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Abstract:  Different  situations  often  require  the  presentation  of  different  types  of 
examples  with  specific  presentation  requirements  about  the  number  of  examples,  the 
sequence  of  presentation,  the  .associated  prompts,  etc.  A  specification  of  the  different 
presentation  requirements  is  particularly  important  in  designing  an  effective  ITS.  A 
categorization  of  example  types  and  their  associated  presentation  requirements  is 
necessary.  In  this  paper,  we  argue  that  examples  must  be  characterized  based  on  the 
context  in  which  they  appear,  and  present  one  such  characterization,  and  describe 
how  these  can  be  effectively  used  by  an  ITS  to  generate  tutorial  descriptions  that 
incorporate  examples. 


Introduction 

Examples  play  an  important  role  in  comprehension,  e.g.,  (Houtz,  Moore,  and  Davis  1973;  Pirolli  1991;  Reder, 
Charney,  and  Morgan  1986),  and  it  is  therefore  important  for  an  intelligent  tutoring  system  to  be  able  to  present 
examples  to  the  learner.  A  large  number  of  examples  can  potentially  be  used  to  illustrate  a  given  point.  However, 
not  all  examples  are  equally  effective  in  all  situations;  some  are  better  than  others  in  specific  contexts,  and  others 
tend  to  illustrate  different  aspects  of  the  same  concept  in  different  ways  and  achieve  different  goals.  Categorizing 
examples  is  useful  because  identifying  a  category  from  which  to  generate  an  example  can  greatly  constrain  the 
number  of  possible  examples  that  can  be  applicable  in  the  given  situation. 

Previous  studies  on  the  categorization  of  examples  include  studies  by  Polya  ( 1 945),  and  Michener  ( 1 978),  on  the 
suitability  of  examples  in  different  situations.  However,  these  categorizations  did  not  explicitly  take  into  account 
the  context  in  which  the  example  was  presented.  Yet,  the  context  of  an  example  affects  its  characterization  and 
usefulness.  To  use  examples  effectively  -  i.e.,  as  an  important  and  a  complementary  part  of  the  overall  description 
-  the  system  must  reason  with  the  constraints  introduced  by  both  the  textual  explanation,  as  well  as  the  examples 
(Mittal  and  Paris  1992;  1993).  This  is  because  both  the  examples  and  the  surrounding  description  affect  each 
other. 

There  are  many  issues  that  must  be  considered  in  selecting  and  presenting  examples,  e.g.,  (Bruner  1966; 
Engelmann  and  Carnine  1982).  In  this  paper,  we  address  the  issue  of  characterizing  the  type  of  examples 
that  appear  in  tutorial  descriptions,  as  this  can  help  a  system  in  choosing  appropriate  examples  to  present.  In  the 
following  sections,  we  describe  previous  work  on  categorizing  example  types,  and  illustrate  bow  the  same  example 
can  be  categorized  in  two  different  categories  if  the  accompanying  description  is  not  taken  into  account.  Finally, 
we  present  our  categorization,  taking  into  consideration  the  context,  based  on  three  orthogonal  dimensions:  the 
information-content,  the  intended-audience  and  the  knowledge-type  of  the  example. 


Previous  Work  on  Categorizing  Examples 

Polya  (1973)  categorized  examples  into  three  categories:  (»')  leading  examples,  (it)  suggestive  examples,  and 
(iii)  counter  examples.  Leading  examples  were  ones  that  contained  mostly  critical1  features  and  very  few 
“irrelevant  features;”  they  were  meant  for  naive  users.  Suggestive  examples  contained  more  variable2  features 
than  leading  examples  and  were  meant  to  “guide  the  student  in  the  correct  direction.”  Counter-examples  were 
negative  examples  that  illustrated  how  instances  were  not  indicative  of  some  concept. 

In  her  work,  Micbener  categorized  examples  into  live  categories  (Michener  1977;  197S):  (i)  introductory 
examples:  perspicuous,  simple  cases,  (ii)  model  examples:  general,  paradigmatic  cases,  (iii)  reference  examples: 
standard,  ubiquitous  cases,  (it>)  counter  examples:  limiting,  falsifying  cases,  and  (v)  anomalous  examples: 
exceptional,  pathological  cases. 

We  believe  that  both  categorizations  suffer  from  two  problems: 

1.  because  they  do  not  explicitly  take  into  account  the  context  of  the  presentation,  the  same  example  can  often 
be  classified  into  different  categories 

2.  the  definition  of  the  category  is  not  clearly  specified;  it  is  therefore  difficult  to  implement  in  a  computational 
system 

Furthermore,  the  two  categorizations  above  did  not  specify  relationships  (if  any)  between  their  different  categories, 
nor  did  they  specify  whether  these  categories  were  mutually  exclusive. 


The  Necessity  of  Categorizing  Examples  based  on  the  Context 

Our  categorization  of  examples  was  driven  by  the  need  to  be  able  to  generate  tutorial  and  explanatory  descriptions 
that  integrate  examples  coherently  in  a  computational  framework.3  In  such  a  framework,  the  system  must  be  able 
to  present  suitable  examples  to  illustrate  the  description  or  the  definition  being  presented.  The  suitability  of  an 
example  is  usually  determined  in  the  context  it  appears  in,  rather  than  in  the  abstract:  it  depends  upon  the  goal  of 
the  description,  what  features  are  being  presented,  where  in  the  overall  description  the  example  appears,  etc. 

Furthermore,  the  suitability  of  the  example  is  also  affected  by  other  examples  around  it.  A  number  of  studies 
on  the  cognitive  effectiveness  of  examples  have  shown  that  the  presentation  order  of  the  examples  plays  an 
important  role  in  user  comprehension,  e.g„  (Carnine  1980;  Litchfield,  Driscoll,  and  Dempsey  1990).  Thus,  the 
appropriateness  of  one  example,  presented  for  the  same  description,  can  be  different,  based  on  other  examples  that 
appear  with  it,  and  where  it  appears.  It  is  therefore  obvious  that  an  example  can  be  categorized  only  in  conjunction 
with  the  context  in  which  it  appears. 

We  shall  now  describe  the  three  dimensions  along  which  we  characterize  an  example  in  context:  the  relationship 
of  the  information  in  the  example  to  that  in  the  context,  the  intended  audience  of  the  example,  and  the  knowledge 
type  being  communicated  by  the  examples. 

The  First  Dimension:  The  relationship  between  the  example  and  the  description 

One  of  the  dimensions  that  an  example  can  be  characterized  along  is  the  relationship  of  the  information  contained 
in  the  example  with  the  information  contained  in  the  accompanying  descriptive  explanation  that  it  illustrates. 
Along  this  dimension,  an  example  can  fall  into  three  categories: 

Positive  Examples:  These  examples  are  instances  of  the 
concept  being  described  and  satisfy  the  properties  of  the 
concept  as  described  in  the  accompanying  description. 
These  examples  must  possess  all  the  critical  features  of 
the  concept  they  illustrate.  Such  examples  are  usually  in 
an  elaborative  role  to  the  information  in  the  description. 
Negative  Examples:  Negative  examples  (or  counter-examples)  are  not  instances  of  the  concept  being  described. 

'Critical  features  are  features  that  are  necessary  for  as  example  to  be  considered  a  positive  example  of  a  concept  Changes  to  a  critical 
feature  cause  a  positive  example  to  become  a  negative  example. 

2Variable  features  are  features  that  can  vary  in  a  positive  example.  Changes  to  variable  features  creates  different  positive  examples. 
3Purther  details  on  this  work  on  the  design  and  implementation  of  a  natural  language  system  capable  of  integrating  examples  and  text  can 
be  seen  in  (Mittal  and  Paris  1992;  1993;  Mittal  1992). 


(AARDVARK)  ;  example  of  a  list 

AARDVARK  ;  not  a  list 

Figure  1:  Two  examples  about  a  list. 


These  are  cases  that  do  not  meet  the  requirements  specified  in  the  accompanying  description,  and  they  play  a 
contrastive  role  in  the  context.  Negative  examples  can  be  very  useful,  because  they  help  rule  out  non-critical 
features  of  a  concept  (Houtz,  Moore,  and  Davis  19731.  For  instance,  the  examples  of  a  list  in  the  programming 
language  lisp  in  Figure  1  illustrate  the  need  for  parentheses  in  a  list.  The  negative  example  conveys  the 
information  that  the  symbol  AARDVARK  by  itself  is  not  sufficient  for  an  instance  to  be  a  list.  By  virtue  of  the  fact 
that  the  only  difference  between  a  positive  and  a  negative  example  is  the  set  of  parentheses,  it  draws  attention 
to  the  fact  that  the  parentheses  are  important  for  something  to  be  a  list.  Thus,  features  in  common  between 
positive  and  negative  examples  can  be  ruled  out  as  sufficient  features,  while  differing  features  are  highlighted  as 
necessary  features  and  thus  become  more  important. 

Anomalous  Examples:  Anomalous  examples  represent  irregular  or  exceptional  cases.  These  are  either:  (») 
instances  of  the  concept  described,  but  not  covered  by  the  description,  or  (it)  those  are  likely  to  be  mis-classified 
by  the  user  (because  of  an  incomplete  description).  Thus,  positive  instances  which  appear  to  be  very  different 
from  other  positive  examples,  or  negative  instances  which  appear  to  be  very  similar  to  positive  examples,  would 
be  classified  as  anomalous  cases.  Anomalous  examples  must  be  presented  with  appropriate  introductory  text,  and 
presented  apart  from  the  other  examples  (Engel man n  and  Camine  1982). 

The  classification  of  an  example  into  either  of  these  categories  depends  upon  the  context  established  by  the 
accompanying  descriptive  explanation.  As  mentioned  previously,  it  is  possible  that  an  example  which  would 
be  classified  as  an  anomalous  example  in  one  context  could  classified  as  a  normal,  positive  example  in  another 
context.  Consider  the  following  description  of  a  list  in  lisp. 

A  left  parenthesis  followed  by  zero  or  more  S -expressions  followed  by  a  right  parenthesis  is  a 
list. 

From  (Shapiro  1986) 


Given  the  above  definition  of  a  1  ist,  the  following  examples  would  classify  as  positive,  negative  and  anomalous 
cases: 


Positive  Examples 

Negative  Examples 

Anomalous  Examples 

(ABCD) 

(1234567) 

(BLUE  SKIES  GREEN  GRASS) 

•THIS-IS-AN-ATOM 

1234567 

’BLUE 

NIL 

(a.b) 

This  categorization  of  examples  could  change  with  another  definition: 


A  list  is  a  coNS-cell  whose  cdr  is  either  the  atom  nil  or  another  list  The  atom  nil  is  the 
identifier  that  represents  the  empty  fist  and  the  boolean  concept  false. 

From  (Steele  Jr.  1984) 


In  this  case,  NIL  becomes  a  positive  example  of  a  list.  Similarly,  a  list  may  be  so  defined  as  to  include  the 
concept  of  a  dotted-list  as  well. 

It  is  clear  that  it  is  difficult,  and  sometimes  impossible,  to  classify  an  example  as  belonging  to  a  certain  category 
without  taking  into  consideration  the  surrounding  contextual  information.  It  is  also  difficult  to  categorize  examples 
as  being  ‘suggestive’  or  ‘model’  or  ‘reference’  without  having  a  complete  definition  of  these  different  categories. 
It  is  also  not  possible  to  label  an  example  ‘positive’  or  ‘negative’  without  knowing  the  definition  it  is  supposed 
to  illustrate  (AARDVARK  is  positive  example  of  an  atom,  but  is  a  negative  example  of  a  list).  In  addition, 
an  example  that  is  ‘anomalous’  in  one  context  can  classify  as  a  positive  example  in  another  context.  Correct 
classification  of  the  examples  is  essential,  because  examples  must  be  presented  in  accordance  with  the  category 
they  happen  to  classify  in.  For  instance,  anomalous  examples  can  cause  great  confusion  in  an  introductory  user 
if  presented  along  with  other  positive  examples.  (Anomalous  examples  should  be  presented  separately  from  the 
regular  examples,  with  a  suitable  introduction  to  notify  the  user  of  the  anomalous  nature  of  such  examples.) 

The  Second  Dimension:  The  intended  audience 

The  second  dimension  that  examples  can  be  characterized  along  is  dictated  by  the  intended  audience  type  of  the 
presentation.  This  is  an  important  constraint  on  the  selection  of  information  to  be  presented  both  in  the  description 


A  list  looks  like  a  sequence  of  objects,  without  commas  between 
them,  enclosed  in  parentheses. 


A  list  always  begins  with  a  left  parenthesis. 
Then  come  zero  or  more  pieces  of  data  (called 
the  elements  of  a  list)  and  a  right  parenthesis. 
Some  examples  of  lists  are: 

(AARDVARK) 

(RED  YELLOW  GREEN  BLUE) 

(2  3  5  11  19) 

(3  FRENCH  FRIES) 

A  list  may  contain  other  lists  as  elements. 
Given  the  three  lists: 

(BLUE  SKY) 

(GREEN  GRASS) 

(BROWN  EARTH) 

we  can  make  a  list  by  combining  them  all  with 
a  parentheses. 

((BLUE  SKY) 

(GREEN  GRASS) 

(BROWN  EARTH) ) 

From  (Touretzky  1984),  page  3S. 

Figure  2:  Introductory  examples  are  usu¬ 
ally  single  featured. 


Appropriately  constructed  lists  can  also  be  used  to  call  functions 
in  LISP.  If  you  type  any  of  the  lists  in  table  2-4  to  LISP,  you  will 
get  an  appropriate  response. 

Table  2-2: 

(12345)  ;  List  of  numbers 

(A  B  C  D)  ;  List  of  symbols 

(#\A  #\B  #\C  #\D)  ;  List  of  characters 
Table  2-3: 

(This  is  (also)  a  list) 

(•this  is  a  string  in  a  list*  -53) 

((Beth  " 555-5834 • )  (Pat  *555-8098-)) 

Table  2-4: 

(SQRT  2) 

(+  2  3) 

(-654) 

Lists  can  be  considered  ways  to  store  data.  For  example,  you 
might  want  to  store  your  inventory  as  a  list,  or  group  together 
names  and  phone  numbers  in  a  list 

From  (Tatar  1987),  page  16. 

Figure  3:  Intermediate  ‘use’  oriented  examples. 


and  the  example.  There  have  beat  many  studies  on  the  need  for  varying  both  the  amount  of  information  and  the 
manner  of  its  presentation,  based  on  the  user,  e.g„  (Paris  1988;  Nwana  1991;  London  1992).  These  studies  have 
demonstrated  that  there  are  significant  differences  in  descriptions  and  examples  meant  for  different  user  types. 

As  we  have  already  mentioned  before,  the  major  short-coming  of  both  the  previous  example  categorizations 
was  due  to  the  fact  that  they  did  not  take  the  accompanying  context  into  account.  In  contrast,  we  consider  both 
the  description  and  the  example  for  categorization.  This  is  essential  in  our  case,  because  the  system  needs  to 
generate  both  the  text  as  well  as  the  example  in  its  explanation.  Often,  even  though  the  examples  tend  to  look 
alike,  the  accompanying  descriptions  are  very  different  for  different  user  types.  For  instance,  Pirolli  found  that  in 
some  domains,  such  as  recursion,  the  examples  presented  to  both  naive  and  advanced  users  were  almost  identical, 
but  their  explanations  were  very  different  (Pirolli  1991;  Pirolli  and  Anderson  198S).  Feldman  and  Klausmdcr 
found  similar  differences  in  the  phrasing  of  definitions  presented  to  fourth  and  eight  grade  students  (Feldman  and 
Klausmeier  1974). 

From  our  analyses  of  naturally  occurring  texts,  we  have  classified  examples  (in  the  context  of  their  accompanying 
descriptions)  into  three  main  classes  -  introductory,  intermediate  and  advanced.  This  classification  constrains  the 
content  and  the  presentation  style  of  the  descriptions  and  the  examples  used  with  them: 


1 .  introductory:  -  users  with  little  or  no  previous  exposure  assumed  for  the  concept;  goal  is  to  learn  about  the 
concept, 

2.  intermediate:  -  users  with  moderate  previous  exposure;  goal  is  to  learn  to  make  use  of  the  concept, 

3.  advanced:  -  users  with  extensive  knowledge;  goal  is  to  clarify  some  point  or  misconception  about  the 
concept. 


Introductory  Users:  Examples  in  introductory  descriptions4  tend  to  be  simple  ones  -  where  ‘simple’  refers  to 
the  fact  that  they  are  usually  single-featured  (or  if  they  have  multiple  features,  sometimes  two,  where  the  two 
features  are  along  two  different  feature  dimensions).  This  has  also  been  reported  in  other  studies,  e.g.,  (Clark  1971; 
Michener  1977;  Canine  1980;  Litchfield,  Driscoll,  and  Dempsey  1990).  In  our  domain  of  lisp  descriptions,  the 
accompanying  description  is  syntactic  or  surface/appearance  oriented.  Anomalous  examples  are  usually  absent, 

4Wc  shall  use  leraa  sucb  u  ‘iatroductory  description*'  to  iodicale  descriptions  meant  for  as  introductory  audience. 


and  if  they  are  presented,  they  are  done  so  after  ail  the  other  examples.  Examples  are  often  introduced  as  soon  as 
the  point  they  illustrate  is  mentioned  in  the  text. 

Consider  for  instance  the  description  in  Figure  2.  The  descriptions  are  centered  around  the  syntax  or  the  surface 
appearance  of  the  list.  The  examples  are  simple  and  illustrate  a  feature  at  a  time  (the  type  of  data  elements, 
except  in  one  case  where  the  type  and  the  number,  two  di  fferent  dimensions  of  variation,  are  illustrated).  Examples 
do  not  always  have  prompts,5  because  the  same  information  is  often  realized  as  sentences  in  the  accompanying 
description. 

Intermediate  Users:  Descriptions  written  for  the  ‘intermediate’  user  (who  is  already  assumed  to  have  introductory 
knowledge)  tend  to  be  more  complex  than  the  ones  for  introductory  users,  in  that  they  include  more  detail  on  how 
the  information  may  be  used  by  the  user.  The  examples  are  not  always  presented  immediately;  if  there  are  a  number 
of  related  points,  these  points  are  stated  first,  before  a  group  of  examples  illustrating  these  points  are  presented. 
The  examples  themselves  are  usually  briefly 
annotated  (with  prompts).  Intermediate  de¬ 
scriptions  contain  a  few  introductory  examples, 
which  are  then  followed  by  typical  uses  of  such 
example  instances,  which  contain  mostly  multi- 
featured  examples.  For  example,  thedescription 
in  Figure  3  describes  bow  a  list  can  be  used 
to  represent  shopping  lists,  store  phone  numbers 
and  write  function  calls. 

Advanced  Users:  Since  the  purpose  of  ad¬ 
vanced  or  reference  materials  is  not  instruction, 
it  is  not  surprising  that  both  the  textual  descrip¬ 
tion  and  the  accompanying  examples  are  very 
different  from  those  in  the  introductory  ones. 

The  documentation  and  the  examples  usually 
occur  in  a  fixed  format,  with  the  examples  fol¬ 
lowing  the  definition  and  the  explanation.  The 
examples  are  not  simple,  single-featured,  but 
tend  to  be  few  and  multi-featured  (typically  three 
to  four  features).  The  examples  are  often  almost 
independent  of  the  textual  description,  with  littlecross-referencing  between  the  two.  This  almost  invariably  results 
in  prompts  bong  used  to  indicate  some  of  the  salient  characteristics  of  the  examples.  Since  the  descriptions  tend 
to  be  comprehensive,  there  are  few  (if  any)  anomalous  examples.  If  there  are  any  anomalous  examples,  they  are 
always  presented.  For  example,  a  description  ofa  list  from  an  advanced,  reference  manual  is  shown  in  Figure  4. 

The  Third  Dimension:  The  Knowledge-Type 

In  addition  to  the  user-type  and  the  example-type  which  can 
be  used  to  constrain  the  possible  choices  that  need  to  be  made 
in  generation,  the  knowledge-type  can  also  be  used  during  the 
generation  process  to  determine  the  appropriate  type  and  sequence 
of  examples  to  be  generated  in  an  explanation.  The  knowledge-type 
refers  to  the  categorization  of  information  into  one  of  three  broad 
classes:  concepts,  relations  or  processes.  There  can  be  significant 
differences  in  the  presentation  of  examples  and  the  accompanying 
descriptions  based  on  whether  the  idea  to  to  be  explained  is  a 
concept,  relation  or  a  process. 

Consider  for  instance  the  concept  ‘list’  (as  described  in  Figure  2) 
and  the  relation  ‘list’  (functions  are  relations  that  hold  between  the  input  parameters  and  the  output  values  of  the 
function),  as  described  in  Figure  5. 

’•Prompts'  an  additional  text  or  marten  associated  with  examples  to  draw  attention  to  specific  features  in  the  examples  (Engelmann  and 
Canine  1982). 


The  list  function  takes  any  number  of  inputs 
and  makes  a  list  of  them  all.  For  example: 


INPUT  to  list 

OUTPUT 

’foo  ’bar  'baz 

— ♦ 

(foo  bar  baz) 

'too 

-» 

(foo) 

’(frob) 

— 

((frob)) 

From  (Touretzky  1984),  p.51 
Figure  3:  Examples  of  a  relation. 


A  list  is  recursively  defined  to  be  eitherthe  empty  list  oraCONS 
whose  CDR  component  is  a  list.  The  CAR  components  of  the 
CONSes  are  called  the  elements  of  the  list  For  each  element  of 
the  list  there  is  a  CONS.  The  empty  list  has  no  elements  at  all. 

A  list  is  annotated  by  writing  the  elements  of  the  list  in  order, 
separated  by  blank  space  (space,  tab,  or  return  character)  and 
surrounded  by  parentheses.  For  example: 

(a  be)  ;  A  list  of  3  symbols 

(2.0s0  (a  1)  #\*)  ;  A  list  of  3  things:a 
;  float,  point  number, 

;  another  list,  and  a 
;  character  object 

The  empty  list  NIL  therefore  can  be  written  (  ),  because  it  is  a  list 
with  no  elements. 

From  Steele  (Steele  Jr.  1984),  p26 
Figure  4:  Reference  documentation  has  few  examples. 


The  concept  list  is  described  as  an  object,  and  examples  of  list  are  instances  which  exemplify  the  term  ‘list’; 
th c  function  list,  on  the  other  hand,  is  described  in  terms  of  its  input  and  output  parameters,  and  examples  of  the 
function  reflect  this  fact.  Similarly,  processes,  which  are  sequences  of  functions  are  described  differently  and 
their  examples  are  often  instances  of  function  parameters  at  every  step  in  the  sequence.  In  generating  examples 
of  relations,  it  is  important  to  keep  in  consideration  that  the  examples  used  as  input-output  parameters  must  be 
known  to  the  hearer.  Also,  since  anomalous  or  pathological  examples  of  concepts  used  as  either  input  or  output 
examples  for  examples  of  relations  often  result  in  anomalous  examples  of  relations,  the  system  must  choose  these 
examples  carefully. 

Examples  of  processes  consist  of  chains  of  events  that  take  place  in  a  particular  order.  The  goal  is  to  communicate 
the  sequence  of  events  and  their  cumulative  effect.  In  case  the  user  does  not  know  about  certain  relations  or 
concepts  involved  in  the  steps  of  the  routine,  the  generator  must  adequately  explain  such  relations  or  concepts  too. 

There  are  two  broad  ways  of  treating  such  ‘gaps’  in  the  bearer’s 
knowledge.  The  missing  information  is  given  as  a  by-note  in  the 
textual  description  accompanying  the  example  if  the  gap  is  a  rel¬ 
atively  minor  one.  If  the  knowledge  required  is  more  than  just  a 
trivial  information  gap,  then  the  generator  should  try  and  postpone 
presenting  examples  using  that  section  of  the  routine  until  the  end 
of  the  sequence.  This  is  to  ensure  that  the  bearer  is  familiar  with 
the  rest  of  the  steps  in  the  sequence  before  the  difficult  examples 
are  encountered. 

Discussion:  The  dimensions  along  which  we  categorize  exam¬ 
ples  are  not  limited  to  the  gradations  mentioned  in  this  paper.  In 
our  framework,  there  are  yet  finer  gradations  which  are  used  by 
the  system  in  making  decisions.  For  instance,  concepts  are  further 
sub-divided  into  single-featured,  multiple-featured,  or  compara¬ 
tive  concepts.  These  finer  gradations  allows  us  to  make  better 
Figure  6:  The  three  dimensions  along  which  decisions  about  both  the  number  of  examples  as  well  as  their  pre¬ 
examples  can  be  categorized  in  context.  sentation  order  in  our  system.  Figure  6  shows  a  representation  of 
- the  three  dimensions  in  this  categorization. 

Applications  to  the  Generation  of  Tutorial  Descriptions 

The  categorization  of  examples  (in  the  context  of 
their  accompanying  description)  that  we  have  out¬ 
lined  is  extremely  useful  in  constructing  a  system 
for  generating  tutorial  descriptions.  Our  major  goal 
is  not  just  the  selection  (or  generation)  of  appropri¬ 
ate  examples  by  themselves,  but  the  generation  of  a 
description  that  integrates  examples  and  text  in  an 
effective  manner.  This  requirement  brings  up  many 
issues  that  may  otherwise  be  not  considered  as  im¬ 
portant:  issues  such  as  the  interaction  between  the 
examples  and  the  description  (how  the  text  changes 
because  of  the  presence  of  examples),  the  placement 
of  the  examples  in  relation  to  the  explanation  (be¬ 
fore,  within  or  after  the  description),  etc. 

Our  current  framework  implements  the  genera¬ 
tion  of  examples  within  a  text-generation  system  by 
explicitly  posting  the  goals  of  providing  examples. 

Our  system  uses  a  planning  mechanism:  given  a 
top  level  communicative  goal,  such  as  (DESCRIBE 
LIST) ,  the  system  finds  plans  capable  of  achieving 
this  goal. 


1.  Determine  if  it  is  a  “higher-order”  noun  (if  it  has 
named  sub-types). 

2.  Pick  positive  examples  that  show  the  different  sub- 
types.  Put  them  together  in  the  sequence. 

3.  Determine  individual  features  that  are  shared  by  the 
all  the  positive  examples  that  have  been  mentioned 
in  Step  #2. 

4.  Construct  negative  examples  that  have  only  one 
of  the  above  critical  features.  This  will  prevent 
generalization  on  the  basis  of  only  that  feature. 

3.  Put  these  together  in  a  sequence  based  on  max¬ 
imum  positive  divergence  and  minimum  negative 
difference  principles. 

Figure  7:  Constructing  Higher-Order  Noun  Sequences 


Plans  typically  post  further  sub-goals  to  be  satisfied,  and  planning  continues  until  primitive  speech  acts  -  i.e., 
directly  realizable  in  English  -  are  achieved.  The  result  of  the  planning  process  is  a  discourse  tree,  where  the  nodes 
represent  goals  at  various  levels  of  abstraction  (with  the  root  being  the  initial  goal,  and  the  leaves  representing 
primitive  realization  statements,  such  as  ( INFORM  . . . )  statements.  In  the  discourse  tree,  the  discourse  goals 
are  related  through  coherence  relations.  This  tree  is  then  passed  to  a  grammar  interface  which  converts  it  into  a 
set  of  inputs  suitable  for  input  to  a  natural  language  generation  system:  Penman  (Mann  1983). 

Examples  are  generated  by  explicitly  posting  a  goal  within  the  text  planning  system:  i.e.,  some  of  the  plan 
operators  used  in  the  system  include  the  generation  of  examples  as  one  of  their  steps,  when  applicable.  This 
ensures  that  the  examples  embody  specific  information  that  either  illustrates  or  complements  the  information  in 
the  accompanying  textual  description.  Issues  such  as  the  number  of  examples  to  be  presented,  the  order  in  which 
they  should  be  presented,  whether  they  should  have  prompts  associated  with  them,  etc.  can  then  be  determined 
in  conjunction  (using  the  constraints  imposed  on  the  selection)  with  the  categorization  of  the  examples  to  be 
presented.  Associated  with  each  gradation  in  our  categorization,  we  have  specific  presentation  heuristics  for  the 
examples  and  their  descriptions.  Figure  7  shows  one  such  set  of  directives  on  the  presentation  of  examples  (for  a 
multi-dimension,  higher-order  concept  for  a  naive  user)  represented  in  our  system. 

The  resulting  discourse  structure  is  then  processed  to  make  final  decisions,  such  as  the  choice  of  lexical  items. 
Finally,  the  completed  discourse  tree  is  passed  to  a  a  system  that  converts  the  INFORM  goals  into  an  intermediate 
form  that  is  accessible  to  Penman,  which  generates  the  desired  English  output. 


Conclusions 

The  categorization  of  examples,  along  with  specific  guidelines  of  when  and  how  different  types  of  examples  should 
be  presented  is  an  extremely  important  issue  in  the  design  of  an  intelligent  tutoring  system.  Our  categorization 
is  a  generalization  of  the  previous  work  by  Mirhener  and  Polya,  and  extends  the  scope  of  the  characterization  to 
take  into  account  the  surrounding  context  of  the  example.  The  categories  along  each  of  the  three  dimensions  that 
we  have  mentioned  are  only  meant  to  illustrate  how  they  affect  the  examples  and  the  text  to  be  presented.  These 
categories  can  be  sub-divided  further  into  smaller  classes  and  specific  presentation  methods  can  be  associated  with 
each  class. 

This  categorization  is  not  specific  to  a  particular  architecture  for  tutoring  systems,  and  can  be  easily  incorporated 
into  any  system  such  as  ceg  (Suthers  and  Rlssland  1988)  or  HYPO  (Ashley  1991).  The  dimensions  can  be  further 
refined  or  modified  if  necessary  to  suit  particular  applications:  for  instance,  recent  work  on  categorizing  dialectical 
examples  (Ashley  and  Aleven  1992)  can  be  easily  incorporated  into  our  framework. 

We  have  implemented  a  system  that  plans  the  presentation  of  coherent  text  and  examples  (Mittal  1992;  Mittal 
and  Paris  1992;  Mittal  and  Paris  1993).  The  plan  operators  (which  select  information  to  be  presented),  also  make 
use  of  the  information  along  the  user-type  and  the  knowledge-type  dimensions  to  structure  the  content  as  well  as  the 
surface  form  of  the  description  appropriately.  Thus,  the  characterization's  modular  nature  allows  the  represented 
information  to  be  shared  among  different  resources  in  the  system. 
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